Image processing apparatus, method, and computer-readable medium for converting frames of image data to higher resolution image data

ABSTRACT

An image processing apparatus characterized by including; an area sensor unit that reads from an original document image a plurality of frames of image data having a shift of less than one pixel, an output resolution acquisition unit that obtains an output image resolution at which resolution the original document image read by the area sensor unit is output, an acquisition frame number control unit that controls a number of frames read by the area sensor unit according to a result of the output resolution acquisition unit, a correction unit that corrects an inclination of the frames of image data controlled by the acquisition frame number control unit, and a high resolution conversion unit that performs an interpolation processing using the plurality of frames of image data whose inclination is corrected by the correction unit to obtain image data in a resolution higher than a resolution during reading.

TECHNICAL FIELD

The present invention relates to an image processing apparatus, acontrol method for the image processing apparatus, and a program and amemory medium for executing the image processing method.

BACKGROUND ART

There exits a technology called “super-resolutionprocessing/super-resolution conversion” that improves a resolution byusing multiple frames of image data in a certain resolution.

The use of this technology enables conversion from a low resolutionimage to a high resolution image, and the high resolution image can beobtained with the same device as a conventional one (“Super-resolutionprocessing With Multiple Digital Image Data” (Ricoh Technical Report No.24, NOVEMBER, 1998)).

The super-resolution technology is widely applied in fields such asmotion picture because multiple frames of image data having minutelydifferent reading positions of an original document image in terms ofsubpixels (a unit smaller than one pixel) of the original document imageare necessary to perform the super-resolution technology.

However, to perform the super-resolution processing, there exists aproblem that the amount of data and the amount of calculations becomelarge because the multiple frames of image data are necessary togenerate one pixel of a high resolution image.

To this end, the amount of calculations has conventionally been reducedby determining the number of frames subjected to the super-resolutionprocessing according to the size of an image area of interest (JapanesePatent Application Laid-Open No. 2006-092450).

However, in the conventional technology as described above, the numberof images subjected to the super-resolution processing is determinedonly in the area of interest. Thus, it is necessary to previouslyensure, in the entire image area, the number of frame image data neededfor the super-resolution processing.

In a case where the super-resolution processing is used for amultifunction apparatus, or an MFP (Multifunction Peripheral), namely,image processing apparatuses, a line sensor is generally used in areader in the multifunction apparatus, a scanner, and the like.

That is, the number of read frames obtained through one readingoperation is one frame.

The above-described reader reads an original document image using agroup of pixel sensors arranged horizontally spaced apart by a distanceof integral multiples of a pixel in a primary scan direction. At thisposition being read, it is impossible to read the original documentimage with a minute (subpixel) shift in the primary scan direction.

To this end, an area sensor is inclinedly arranged in the apparatus, sothat one reading operation allows obtaining image data with the minuteshift in the primary scan direction and/or a secondary scan direction atthe position of the pixels being read.

However, at this occasion, multiple frames of low resolution frame imagedata having been read are used regardless of output conditions of theimage data.

Thus, there exists a problem that the amount of data and the amount ofcalculations increase because the low resolution frame image data arememorized more than necessary and that the super-resolution processingis performed using them.

DISCLOSURE OF THE INVENTION

The present invention is made in consideration of the above-describedproblems, and it is the object of the present invention to provide animage processing apparatus and an image processing method for the imageprocessing apparatus that enable reducing the amount of data and theamount of calculations in an MFP system using the super-resolutionprocessing.

To achieve the above-described object, the image processing apparatusaccording to the present invention has:

an area sensor unit that reads from an original document image aplurality of frames of image data having a shift of less than one pixel;

an output resolution acquisition unit that obtains an output imageresolution at which resolution the original document image read by thearea sensor unit is output;

an acquisition frame number control unit that controls a number offrames read by the area sensor unit according to a result of the outputresolution acquisition unit;

a correction unit that corrects an inclination of the frames of imagedata controlled by the acquisition frame number control unit; and

a high resolution conversion unit that performs an interpolationprocessing using the plurality of frames of image data whose inclinationis corrected by the correction unit to obtain image data in a resolutionhigher than a resolution during reading.

According to the present invention, an amount of low resolution imagedata needed for the super-resolution processing can be obtainedaccording to conditions in which the image is output. Thus, the amountof data and the amount of calculations can be reduced.

Further features of the present invention will become apparent from thefollowing description of exemplary embodiments with reference to theattached drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic diagram of an image processing apparatus.

FIG. 2 is a figure showing a structure of a reading section of an imageprocessing apparatus 1.

FIG. 3 is a block configuration diagram showing a controller structureof the image processing apparatus 1.

FIG. 4 is a block configuration diagram showing an internal structure ofa scanner image processor 312.

FIG. 5 is a figure showing an example of an image obtained with ascanner 11.

FIG. 6 is a block configuration diagram showing an internal structure ofa printer image processor 215.

FIG. 7 is a figure showing a structure of an area sensor.

FIG. 8 is an original document image read by the area sensor.

(a), (b), (c), (d) and (e) of FIG. 9 are figures showing a method forobtaining line image data.

(a), (b), (c), (d) and (e) of FIG. 10 are figures showing the method forobtaining the line image data.

(a), (b), (c), (d) and (e) of FIG. 11 are figures showing the method forobtaining the line image data.

(a), (b), (c), (d) and (e) of FIG. 12 are figures showing the method forobtaining the line image data.

(a) and (b) of FIG. 13 are image data read with the line sensor in thearea sensor.

(a) and (b) of FIG. 14 are configuration diagrams in a case where thearea sensor is obliquely mounted.

(a), (b), (c), (d) and (e) of FIG. 15 are figures showing a method forobtaining the line image data with the inclined area sensor.

(a), (b), (c), (d) and (e) of FIG. 16 are figures e showing the methodfor obtaining the line image data with the inclined area sensor.

(a), (b), (c), (d) and (e) of FIG. 17 are figures showing the method forobtaining the line image data with the inclined area sensor.

(a) and (b) of FIG. 18 are image data read with a line sensor in theinclined area sensor.

FIG. 19 is a flowchart for describing an outline of operation performinga super-resolution processing mode setting processing according to thefirst embodiment.

FIG. 20 is a schematic diagram about the detail of the super-resolutionprocessing.

FIG. 21 is a schematic diagram about the detail of the super-resolutionprocessing.

FIG. 22 is a flowchart for describing an outline of operation performingan output mode acquisition processing shown in FIG. 19.

FIG. 23 is a GUI displaying an example of an operation section forsetting the output mode acquisition processing shown in FIG. 22according to the first embodiment.

FIG. 24 is a figure showing a dividing method of the area sensor.

FIG. 25 is a flowchart for describing an outline of operation performingthe output mode acquisition processing shown in FIG. 19 according to thesecond embodiment.

FIG. 26 is a GUI displaying an example of the operation section forsetting the output mode acquisition processing shown in FIG. 25according to the second embodiment.

FIG. 27 is a flowchart for describing an outline of operation performingthe output mode acquisition processing shown in FIG. 19 according to thethird embodiment.

FIG. 28 is a GUI displaying an example of the operation section forsetting the output mode acquisition processing shown in FIG. 27according to the third embodiment.

FIG. 29 is a flowchart for describing an outline of operation performingthe output mode acquisition processing shown in FIG. 19 according to thefourth embodiment.

FIG. 30 is a GUI displaying an example of the operation section forsetting the output mode acquisition processing shown in FIG. 29according to the fourth embodiment.

FIG. 31 is a figure showing the dividing method of the area sensorexisting in the scanner 11 according to the fifth embodiment.

BEST MODES FOR CARRYING OUT THE INVENTION

Embodiments of the present invention will be hereinafter described withreference to figures.

First Embodiment

An embodiment of the present invention will be described. In the presentembodiment, an image processing apparatus having a color scanner is asubject matter, and a technique for generating a high resolution imageusing an area sensor will be described.

<About External Appearance of Image Processing Apparatus>

An external appearance of an image processing apparatus 1 is shown inFIG. 1. The image processing apparatus 1 is divided into a scanner 11for reading an original document image, a printer 12 for reproducingread image data, and an operation section 13 for designating variousoperation settings of the image processing apparatus 1. The scanner 11converts information of the original document image into an electricsignal by inputting into a CCD a reflected light obtained by exposingand scanning an image on the original document image. The scanner 11further converts the electric signal into a brightness signal includingR, G, B colors, and outputs this brightness signal as image data to acontroller 20 later described in FIG. 3.

It should be noted that the original document image is set on a tray 14of a document feeder 15. When a user instructs to start reading with anoperation section 13, an original document image-reading instruction isgiven from the controller 20, later described in FIG. 3, to the scanner11.

Upon receiving this instruction, the scanner 11 perform a readingoperation of the original document image by feeding, one by one, theoriginal document images from the tray 14 of the document feeder 15.

It should be noted that a reading method of the original document imageshould not necessarily be an automatic feeding method with the documentfeeder 15, but may be a method for scanning the document upon placingthe original document image on a glass surface, not shown, and moving anexposure section.

The printer 12 is an image processing device for forming image data,received from the controller 20, on a sheet. It should be noted thatalthough an image processing method of the present embodiment is anelectrophotographic method using a photosensitive drum and aphotosensitive belt, the present invention is not limited thereto. Forexample, an inkjet method for printing by propelling ink onto a sheetthrough a minute nozzle array may also be applied. The printer 12 isarranged with multiple paper cassettes 17, 18, 19 to allow selection ofdifferent sheet sizes and different sheet orientations. The sheetsfinished with printing are discharged to a discharge tray 16.

(Structure of Reading Section of Image Processing Apparatus)

FIG. 2 is a figure showing a structure of a reading section of amultifunction apparatus to which the present embodiment is applied.

In the figure, the apparatus comprises a reading section 201, and an ADFsection 202 which has an ADF function for holding an original document203 and conveying the original document to a document reading positionduring successive reading.

The apparatus further comprises a glass plate, or a platen, which mountsthe original document 203 for reading of an original document image onthe original document.

The apparatus still further comprises a unit 205 which includes areading device for reading the original document image 203, and includesa device for photographing the original document image.

A light source 206 is provided, and a white light source such as a xenontube is used therefor.

Mirrors 207 to 211 are provided which have a role to transmit to aphotographing element a reflected light of a light emitted from thelight source 206 to the document surface.

A lens 212 is provided which condenses the reflected light coming fromthe original document image and reflected by the mirror 212 into thewidth of a photographing element 213.

<Detailed Description of Controller>

FIG. 3 is a block diagram for describing the detail of a structure ofthe controller 20 existing in the image processing apparatus 1.

The controller 20 is electrically connected to the scanner 11 and theprinter 12, and on the other hand, is connected to an external apparatusvia a LAN 21 or a WAN 22. Thus, the controller 20 can input and outputimage data and device information.

A CPU 301 controls accesses to various connected devices based on acontrol program memorized in a ROM 303. The CPU 301 also controlsvarious processings performed in the controller 20.

A RAM 302 is a system work memory for allowing the CPU 301 to operate,and is the memory for temporarily memorizing the image data. The RAM 302includes an SRAM for holding the memorized content even after power-offand a DRAM for erasing the memorized content after power-off.

The ROM 303 contains a boot program for the apparatus. An HDD 304 is ahard disk drive that can contain system software and the image data.

An operation section I/F 305 is an interface section for connecting asystem bus 310 and the operation section 13. The operation section I/F305 not only receives from the system bus 310 the image data to bedisplayed on the operation section 13 and outputs the image data to theoperation section 13, but also outputs information input from theoperation section 13 to the system bus 310.

A network I/F 306 is connected to the LAN 21 and the system bus 310, andinputs and outputs information. A modem 307 is connected to the WAN 22and the system bus 310, and inputs and outputs information. A binaryimage rotator 308 converts an orientation of the image data prior totransmission. A binary image compressor/expander 309 converts aresolution of the image data prior to transmission into a predeterminedresolution or a resolution according to capabilities of an opponent. Itshould be noted that methods such as JBIG, MMR, MR, and MH are used forcompression and extension. An image bus 330 is a transmission path forsending and receiving the image data, and is made of a PCI bus or anIEEE 1394.

A scanner image processor 312 performs correction, processing, andediting on the image data received from the scanner 11 via the scannerI/F 311. It should be noted that the scanner image processor 312 makes adetermination as to whether the received image data is a color documentor a black and white document and whether a character document or apicture document. Then, the determination result is attached to theimage data. Such accompanying information is called attribute data. Theprocessings performed by this scanner image processor 312 will bedescribed later in detail.

A compressor 313 receives the image data, and divides this image datainto units of blocks in 32 pixels by 32 pixels. It should be noted thatthis image data in 32 pixels by 32 pixels is called tile data. An areaof an original document (paper medium not yet being read) correspondingto this tile data is called a tile image. It should be noted thataverage brightness information in this block in 32 pixels by 32 pixelsand a coordinate position on the original document of the tile image areattached as header information to the tile data. Furthermore, thecompressor 313 compresses the image data including multiple tile data.An expander 316 expands the image data including the multiple tile data,and thereafter, performs raster expansion and transmits the image datato a printer image processor 315.

The printer image processor 315 receives the image data sent from theexpander 316, and performs an image processing on the image data whilereferring to attribute data attached to this image data. The image datahaving been subjected to the image processing is output to the printer12 via a printer I/F 314. The processing performed by this printer imageprocessor 315 will be later described in detail.

An image converter 317 performs a predetermined conversion processing onthe image data. Processors as shown below constitute this processor.

An expander 318 expands the received image data. A compressor 319compresses the received image data. A rotator 320 rotates the receivedimage data. A scaler 321 performs a resolution conversion processing(for example, from 600 dpi to 200 dpi) on the received image data. Acolor space converter 322 converts a color space of the received imagedata. This color space converter 322 can use a matrix or a table toperform a known background color removal, to perform a known LOGconversion processing (RGB to CMY), and to perform a known output colorcorrection processing (CMY to CMYK). A binary multi-value converter 323converts the received two-level grayscale image data into 256-levelgrayscale image data. In reverse, a multi-value binary converter 324converts the received 256-level grayscale image data into two-levelgrayscale image data through a method such as error diffusionprocessing.

A combiner 327 combines the received two image data to generate oneimage data. It should be noted that the two image data are combinedthrough methods such as a method for making an average value ofbrightness values of pixels to be combined a combined brightness valueand a method for making a brightness value of a brighter pixel inbrightness level a brightness value of a combined pixel. In addition, itmay be possible to employ a method for making a darker one the combinedpixel. Furthermore, it may also be possible to employ a method fordetermining the combined brightness value through logical OR operation,logical AND operation, and exclusive OR operation of the pixelssubjected to combining. Either of these combining methods is a knownmethod. A thinning section 326 converts the resolution by thinning outpixels in the received image data to generate ½, ¼, ⅛ image data. Amoving section 325 adds a blank portion to the received image data anddeletes the blank portion therefrom.

A RIP 328 receives intermediate data generated based on a PDL code datatransmitted from a printer server (not shown), generates bit map data(multiple values), and compresses the bit map data with a compressor329.

<Detailed Description of Scanner Image Processor 312>

FIG. 4 shows an internal structure of the scanner image processor 312.

The scanner image processor 312 receives the image data including thebrightness signal of RGB, each in 8-bit, via the scanner I/F 311 fromthe scanner 11. This brightness signal is converted by a maskingprocessor 401 into a standard brightness signal not relying on a filtercolor of the CCD.

A filter processor 403 arbitrarily corrects a spatial frequency of thereceived image data. This processor uses, for example, 7 by 7 matrix toperform a calculation processing on the received image data. In themeantime, on a copier, a user can select a character mode, a photographymode, a character/photography mode by operating the operation section13. Herein, in a case where the user selects the character mode, thefilter processor 403 applies a character filter on the entire imagedata. In a case where the photography mode is selected, a photographyfilter is applied on the entire image data. In a case where thecharacter/photography mode is selected, the filter is adaptivelyswitched for each pixel according to a later-described characterphotography determination signal (a portion of the attribute data). Thatis, a determination is made for each pixel as to whether the photographyfilter is applied or the character filter is applied. It should be notedthat the photography filter is set with such coefficients that only highfrequency components are smoothed. This is to render unevenness of theimage inconspicuous. On the other hand, the character filter is set withsuch coefficients that edge emphasis is strongly applied. This is toincrease sharpness of characters.

A histogram generator 404 samples brightness data of each pixelconstituting the received image data. To describe in more detail,brightness data in a rectangular area enclosed by a start point and anend point designated in each of the primary scan direction and thesecondary scan direction are sampled at a constant pitch in the primaryscan direction and the secondary scan direction. Then, histogram data isgenerated based on the sampling result. The generated histogram data isused to predict a background level when the background color removal isperformed. An input side gamma corrector 405 converts, with a table, thegenerated histogram data into brightness data having nonlinearcharacteristic.

A color monochrome determination section 406 makes a determination as towhether each pixel constituting the received image data is chromaticcolor or achromatic color, and the determination result is attached as acolor monochrome determination signal (a portion of the attribute data)to the image data.

A character photography determination section 407 makes a determination,based on a pixel value of each pixel and pixel values of surroundingpixels around each pixel, as to whether each pixel constituting theimage data is a pixel constituting a character, a pixel constituting ahalftone dot, a pixel constituting a character in the halftone dot, anda pixel constituting a solid image. A pixel not applicable to either ofthem is a pixel constituting a white area. Then, this determinationresult is attached as a character photography determination signal (aportion of the attribute data) to the image data.

A super-resolution processor 402 performs a super-resolution processingon the received image data. It should be noted that on a copier, a usercan select a processing mode of the super-resolution processing byoperating the operation section 13.

The processing modes of the super-resolution processing will bedescribed later in detail. There exist conditions for performing thesuper-resolution processing.

That is, it is necessary to have multiple frames of image data of theoriginal document image having minute shifts of reading positions in theprimary scan direction and/or the secondary scan direction with respectto image data of the original document image having been read first at asensor resolution of the reader.

Namely, it is necessary to have multiple, successive frames of imagedata whose positions of the original document read by the sensor areshifted little by little in the primary scan direction and/or thesecondary scan direction from the image data serving as the reference.

In addition, when these multiple frames of image data are read, theshifts of the reading positions of the original document image, existingbetween the image data obtained with adjacent sensors, need to be lessthan one pixel (subpixel) in the primary scan direction and/or thesecondary scan direction.

This shift of the reading position may be a shift less than one pixelthat remains as a result of a shift correction performed on a positionalshift in integral multiples.

Hereinbelow, data read when the original document image including onescreen (frame) is scanned, namely, data constituting this one screen(frame) of the original document image, is hereinafter called “frameimage data”.

A position of a pixel being read on the original document image iscalled “phase”.

A shifting of this phase is referred to as “phase is shifted”, and theshift of the read pixel is called “phase shift”.

A low resolution used here is not limited to 300 dpi, but means aresolution of an image that the apparatus outputs in normal printing.

The primary scan direction is a direction perpendicular to a directionthat a unit 205 moves with respect to the original document image whenthe scanner reads the original document image placed on a documenttable.

In FIG. 8, the primary scan direction is shown by arrow A, which is alateral direction of this read original document image.

Similarly, the secondary scan direction is a direction horizontal to themoving direction of the unit 205.

In FIG. 8, the secondary scan direction is shown by arrow B, which is alongitudinal direction of this read original document image.

In the present embodiment, the area sensor is obliquely arranged, sothat multiple images having phase shifts with respect to the primaryscan direction and the secondary scan direction can be obtained for eachRGB channel.

FIG. 5 shows examples of the image obtained through the presentembodiment.

Images 501 to 504 are obtained in a state where the phase is shifted ineach of the primary scan direction and the secondary scan direction.

(Area Sensor)

In the present embodiment, the sensor for reading the image includes thearea sensor.

The area sensor is a photographing element applied to a digital cameraand the like. In contrast to the above-described sensors in units oflines, the area sensor has two-dimensionally arranged pixel sensors forreading data.

FIG. 7 is a figure showing a structure of the area sensor. In FIG. 7,numeral 701 denotes an area sensor device.

Pixel sensors 702 in the area sensor 701 includes pixel sensors having Hpixels in a long side and L pixels in a short side.

This one pixel may be a pixel sensor supporting colors including RGBmade by dividing the pixel sensor for this one pixel into four. It mayalso be possible to set H pixels=L pixels (long side=short side).

The resolution of this area sensor is determined by a distance N betweenthe pixel sensors.

The area sensor used in a high resolution digital camera includesextremely many pixels as long-side pixel sensors and shorter side pixelsensors. For example, there exists a digital camera of ten millionpixels class that has 3,800 pixels as the long-side pixel sensors and2,800 pixels as the short-side pixel sensors.

Generally, when the area sensor is used for a camera, the area sensorphotographs by capturing input image data as a two-dimensional area.

That is, in one photographing, the two-dimensionally arranged pixelsensors are used to photograph. When the area sensor device is attachedto the reader, the pixel sensors are arranged without inclination sothat the photographed original document image does not have a distortionin the lateral direction and the longitudinal direction.

Thus, the arrangement is made to eliminate any shift in an obliquedirection in a case where the photographed image is reproduced.

For example, in a case where the area sensor is attached in a commonlyused camera, the image data read by the pixel sensors on a line shown bya black border 703 are image data constituting an uppermost portion of aphotographed object.

In such occasion, the read image data is caused to be not inclined in adirection constituting a line.

Similarly, image data read by the pixel sensors on a line shown by ablack border 704 is image data at a position different from the positionof the photographed object read by numeral 703, namely, image data at aposition below in a vertical direction. Thus, numeral 705 is the imagedata at a position four lines below in the vertical direction withrespect to the photographed position read by numeral 703.

In this way, when the area sensor of the digital camera is used, theimage data is photographed as the two-dimensional area. Thus, the pixelsensors constituting the area sensor are all photographing differentpositions of the photographed object.

However, the usage method of the area sensor in the apparatus used inthe present embodiment is different from a usage method in theabove-described digital camera.

First, the area sensor as shown in FIG. 7 is attached to an arrangementposition serving as a reference of the reader.

When the original document image is placed at a designated position onthe platen 204 in FIG. 1, condensed on a sensor is a reflected light ofa light emitted to the original document image from a light sourcescanning parallel under the original document image in the samedirection as the longitudinal direction of the original document image.This reflected light is incorporated in such manner that the reflectedlight does not incline with respect to the sensor.

The reflected light, as one line of image data, obtained through theparallel scanning of the light source is condensed parallel to thelateral direction (long-side direction) of the sensor shown in FIG. 7.

Thus, the sensor is arranged at a position capable of incorporating theoriginal document image substantially without inclination.

As described above, the arrangement position of the sensor for realizingthe output of the original document image is made “a referencearrangement position” of the sensor.

For the purpose of simplicity of the description, it is assumed in thisdescription that the sensor includes the pixel sensors having 20 pixelsin the long-side direction and 10 pixels in the short-side direction.

Needless to say, a structure that the long-side direction=the short-sidedirection may also be possible.

It should be noted that the number of the above-described pixel sensorsis for the purpose of describing usage and structure of the area sensoraccording to the present embodiment, and is not limited to the number ofthe pixel sensors shown in the figure.

In reality, needless to say, it may also be possible to structure withas many pixel sensors as used in the digital camera.

An original document image 103 placed on the platen 204 is read bydriving the reading unit 205 including the area sensor 213 in the readerin a direction of the arrow illustrated in FIG. 2.

That is, a reading operation is performed by handling the reading linesensors 704 and 705, i.e., a group of pixel sensors, just like theabove-described line sensor.

Next, described is how the original document image read by the readingline sensors 704 and 705 becomes.

In this description, it is assumed that FIG. 8 is the original documentimage to be read.

What is shown in a lattice in the figure corresponds to the resolutionof the pixel sensors constituting the reading line sensor 704 or 705.

When the reading unit 205 drives and moves under the document table inthe secondary scan direction, the image data input to the reading linesensors 704 and 705 are successively read.

That is, among the original document image, a portion corresponding to aline width equivalent to the position of the reading unit 205 is readfrom moment to moment.

A process for reading this original document image will be described.

When the reading unit 205 moves under the document table in thesecondary scan direction, shaded areas of the original document imageshown in (a) of FIG. 9, (a) of FIG. 10, (a) of FIG. 11, (a) of FIG. 12are exposed to the light from the light source.

First, at a certain instance, the shaded area in (a) of FIG. 9 isexposed to the light from the light source.

Then, the area sensor detects the light to detect the original documentimage at a line width portion, i.e., a portion exposed to the light.

For example, at this moment, the line sensor 704 detects the image dataas shown in (b) of FIG. 9.

At the same time, the line sensor 705 detects image data as shown in (c)of FIG. 9.

The reason why there simultaneously exists a shift between the readingpositions of the two image data is that the two line sensors arearranged with a physical distance in the short-side direction.

Then, the read original document image is treated as different imagedata by each reading line sensor, and the image data are separatelymemorized to memory media such as memories as shown in (d) and (e) ofFIG. 9.

Next, as the sensor unit 205 moves and the light source moves, theposition of the original document image detected by the line sensorchanges as shown in (a) of FIG. 10. Then, the line sensor 704 detects animage as shown in (b) of FIG. 10, and the line sensor 705 detects animage as shown in (c) of FIG. 10.

The read original document images are treated as different image data byeach reading line sensor, and the image data are separately memorized tomemory media such as memories as shown in (d) and (e) of FIG. 10.

Similarly, when the position as shown in (a) of FIG. 11 is read, imagedata as shown in (b) and (c) of FIG. 11 are memorized to memory mediasuch as memories as shown in (d) and (e) of FIG. 11.

On the other hand, when the position as shown in (a) of FIG. 12 is read,image data as shown in (b) and (c) of FIG. 12 are memorized to memorymedia such as memories as shown in (d) and (e) of FIG. 12.

Eventually, the entire original document image is exposed to the lightfrom the light source, and each line sensor reads the image data at eachposition.

Then, the read image data are successively stored to the memories, andmultiple image data with the shift equivalent to one pixel in thesecondary scan direction as shown in each of (a) and (b) of FIG. 13 canbe obtained.

These image data having the shift in the secondary scan direction canobtain as many frame image data as the number of line sensors includingthe group of area sensors.

Thus, in a case where the pixel sensors, two-dimensionally arranged toread the image, is used as the area sensor, multiple frames of frameimage data whose phases are continuously shifted in the secondary scandirection are obtained through one reading operation.

Next, the usage method of the area sensor in the apparatus used in thepresent embodiment is described.

First, the area sensor as shown in FIG. 7 is inclinedly mounted to thereader.

An example of a mounting form of the area sensor according to thepresent embodiment is shown in (a) of FIG. 14.

Numeral 1401 denotes the area sensor device.

Numeral 1402 denotes the image sensor, and it is assumed in thisdescription that the image sensor includes the pixel sensors having 20pixels in the long-side direction and 10 pixels in the short-sidedirection.

Then, the area sensor is mounted in such manner that the area sensorinclines with respect to the reference arrangement position.

That is, the area sensor is arranged at an angle θ with respect to theline sensor arranged at the lowermost of the area sensor when arrangedat the reference arrangement position as shown in FIG. 7.

The positions of the constituting pixel sensors are indicated assuming atop left end as the origin point, the long-side direction as xdirection, and the short-side direction as y direction.

That is, the coordinate of the top left end is (x, y)=(0, 0), and thecoordinate of the top right end is (x, y)=(19, 0).

Similarly, the coordinate of the bottom left end is (x, y)=(0, 9), andthe coordinate of the right bottom end is (x, y)=(19, 9).

Numeral 1403 denotes a group of one line of the pixel sensorsconstituting an area sensor 1401, and specifically, includes 20 pixelsensors constituting the long-side direction.

That is, numeral 1403 includes the pixel sensors at the coordinatepositions (0, 4), (1, 4), (2, 4), . . . (19, 4).

It should be noted that in the below description, multiple pixel sensorsenclosed by the numeral 1403 are called a reading line sensor 1403.

Similarly, numeral 1404 includes the pixel sensors at the coordinatepositions (0, 5), (1, 5), (2, 5), . . . (19, 5), and is referred to as areading line sensor 1404 in the below description.

In the present embodiment, the reading unit 205 including the areasensor 213 in the reader is caused to drive in the arrow directionillustrated in FIG. 2 to read the original document image placed on thedocument table 204.

That is, the reading line sensors 1403 and 1404, i.e., the group ofpixel sensors, are treated as the line sensor to perform the readingoperation as described above.

Next, described is how the original document image read by the readingline sensors 1403 and the reading line sensor 1404 becomes.

In this description, FIG. 8 is the original document image to be read.

That is, this original document image corresponds to the originaldocument image 203 in FIG. 2.

What is shown in a lattice in the figure corresponds to the resolutionof the pixel sensors constituting the reading line sensor 1403 or 1404.

The original document image is read as shown in above-described (a) to(e) of FIG. 9 to (a) and (b) of FIG. 13, but the image data inclined atthe angle θ are obtained because of the inclination θ.

For example, originally, if the area sensor were not inclined, aposition shown by a shaded area in (a) of FIG. 15 would be read.However, due to the inclination of the area sensor, the line sensor1403, 1404 detect image data as shown in (b) and (c) of FIG. 15.

Then, these image data are respectively memorized to memory media suchas memories as shown in (d) and (e) of FIG. 15 while the image data arestill inclined.

Similarly, a position shown by a shaded area in (a) of FIG. 16 is readwhile the sensor unit 205 moves and the light source moves. At thismoment, the line sensors 1403, 1404 detect image data as shown in (b)and (c) of FIG. 16.

Then, these image data are respectively memorized to memory media suchas memories as shown in (d) and (e) of FIG. 16.

In addition, a position shown by a shaded area in (a) of FIG. 17 is readwhile the reading unit moves in the secondary scan direction and thelight source moves. At this moment, the line sensors 1403, 1404 detectimage data as shown in (b) and (c) of FIG. 17.

Then, these image data are respectively memorized to memory media suchas memories as shown in (d) and (e) of FIG. 17.

The image data eventually having been detected and read by the linesensors 1403, 1404 are data shown in (a) and (b) of FIG. 18, and eitherof the image data is read as image data inclined at the angle θ.

At this moment, a direction shown by arrow (A) in (a) and (b) of FIG. 18is called the primary scan direction, and a direction shown by arrow (B)is called the secondary scan direction.

In contrast, a direction shown by arrow (C) is called a lateraldirection of the read image data. On the other hand, a direction shownby arrow (D) is called a longitudinal direction of the read image data.

As shown in (a) of FIG. 14, the reading line sensor 1403 and the readingline sensor 1404 are physically shifted by an amount equivalent to onepixel sensor in the short-side direction.

Thus, the pixel sensors constituting the reading line sensor 1403 andthe pixel sensors constituting the reading line sensor 1404 have a phaseshift in the long-side direction.

For example, the position of the pixel sensor at a coordinate (x,y)=(15, 4) of the reading line sensor 1403 and the position of the pixelsensor at a coordinate (x, y)=(15, 5) of the reading line sensor 1404are shifted in the y axis direction, i.e., the short-side direction, byan amount equivalent to y=1.

This shift brings about a shift equivalent to Δβ in the secondary scandirection.

On the other hand, the position in the x axis direction, i.e., thelong-side direction, is completely the same, namely, x=15.

However, due to the inclination angle θ, the phase is shifted by aminute amount Δα within the subpixel in the horizontal direction of thereference arrangement position.

That is, a phase shift in minute units occurs by inclining the areasensor even in a case of the pixel sensor at the same position in thelong-side direction, i.e., the x axis direction, of the reading linesensor. This phase shift relies on the inclination angle.

Thus, the image data read by the reading line sensor defined in the areasensor 213 becomes image data having phase shifts different for eachline sensor.

Specifically, the phase of the read image data in (a) of FIG. 18 and thephase of the read image data in (b) of FIG. 18 are not only shifted byΔβ in the secondary scan direction but also shifted by Δα in the primaryscan direction.

In the above-described description, it is assumed that there exist tworeading line sensors (reading line sensors 1403, 1404). But it is notlimited thereto.

It may also be possible to increase the pixel sensors constituting thearea sensor 113 in the x axis direction, so that many area sensors arearranged.

That is, it may be possible to arrange as many reading area sensors asthe number of pixels lining up in the x axis direction constituting thearea sensor 213.

The number of the arranged reading line sensors is equal to the numberof the image data obtained per one reading operation.

That is, if thirty lines of reading line sensors are arranged in thearea sensor 213, thirty read images each having a unique phase shift canbe obtained through one reading operation.

By inclining the area sensor, one scanning of the original documentimage allows obtaining multiple frame image data whose positional shiftof the original document image read by the sensors adjacent to eachother in the short-side direction is less than one pixel.

On the other hand, the sensors may be arranged as shown in (b) of FIG.14.

The long-side direction is the same direction as a horizontal directionin the reference arrangement position. However, the short-side directionis inclined with respect to the reference arrangement position.

Also in this case, similar to (a) of FIG. 14, one scanning of theoriginal document image allows obtaining the frame image data whosepositional shift read by the sensors adjacent to each other in theshort-side direction is less than one pixel in the primary scandirection and/or the secondary scan direction.

That is, in the area sensor including multiple sensors, any sensor willdo as long as the sensor allows obtaining the frame image data whosepositional shift read by the sensors adjacent to each other in theshort-side direction is less than one pixel in the primary scandirection and/or the secondary scan direction when a scanning positionmoves parallel relative to the original document image.

On the other hand, both of the angles θ in (a) of FIG. 14 and the angleθ′ in (b) of FIG. 14 will do as long as the angles are within a rangecapable of obtaining the frame image data whose positional shift read bythe sensors adjacent to each other in the short-side direction throughone scanning of the original document image is less than one pixel inthe primary scan direction and/or the secondary scan direction.

Furthermore, the number of the frame image data obtained in theshort-side direction of the sensor may be increased by increasing thenumber of times of readings in the secondary scan direction duringreading of the original document image and by increasing the number oftimes of samplings per unit time.

<Detailed Description of Printer Image Processor 315>

FIG. 6 shows an internal structure of the printer image processor 315.

A background skipping processor 601 uses the histogram generated by thescanner image processor 312 to skip (remove) a background color of theimage data.

A monochrome generator 602 converts color data into monochrome data. ALog converter 603 performs a brightness density conversion. This Logconverter 603 converts, for example, the RGB input image data into CMYimage data.

An output color corrector 604 performs an output color correction. Forexample, the CMY input image data is converted into CMYK image datausing a table and a matrix.

An output side gamma corrector 605 performs a correction so that asignal value input into this output side gamma corrector 605 becomesproportional to a reflection density value after copy and output.

A halftone corrector 606 performs a halftone processing according to thenumber of levels of halftone of the printer that outputs. For example,the halftone corrector 606 performs, on the received multilevel halftoneimage data, a conversion into binary and a conversion into 32-value.

It should be noted that each processor in the scanner image processor312 and the printer image processor 315 can output the received imagedata without performing each processing. Causing a processor to allowdata to pass therethrough without performing any processing in this wayis hereinafter referred to as “pass through a processor”.

<Processing Mode Setting of Super-resolution Processing>

A processing mode setting according to the present embodiment will behereinafter described in detail using FIG. 19.

It should be noted that the area sensor shown in (a) of FIG. 14 is usedin the present embodiment. The image processing apparatus is used thathas the area sensor capable of obtaining 100 frames of low resolutionframe image data equivalent to 100 dpi when the original document imageis read. This image processing apparatus can generate a 200 dpi highresolution image from four frames of low resolution frame image data inthe super-resolution processing.

Similarly, the image processing apparatus can generate a 300 dpi imagefrom 10 frames of low resolution frame image data.

In addition, the image processing apparatus can generate a 600 dpi highresolution image from 40 frames of low resolution frame image data.

The image processing apparatus can generate a 1200 dpi high resolutionimage from 100 frames of low resolution frame image data.

It should be noted that the number of frames of low resolution imagedata needed for obtaining a desired resolution is not limited to theabove-described number of frames of low resolution image data. Forexample, a 1000 dpi high resolution image may be generated from 50frames, and a 500 dpi high resolution image may be formed from 50frames.

This depends on a capability of the image processing apparatus.

On the other hand, not all obtained low resolution images can be used.

For example, an image whose shift of the reading pixel position of theframe image data read by the adjacent sensors, i.e., phase shift, is onepixel or more cannot be used for the super-resolution processing.

Such low resolution frame image data is not counted as the obtained lowresolution frame image data.

FIG. 19 is a figure for describing an outline of operation performing asuper-resolution processing-mode setting processing. A control programrealizing the processing shown in FIG. 19 is contained in the ROM 303 asdescribed above, and is executed by the CPU 301.

First, in step S1901, an instruction of an output mode is received fromthe user via a user interface (hereafter called “UI”). In response, theUI having received this instruction sends this instruction to the CPU301 via the operation section I/F, and the CPU obtains the output mode.

Next, the CPU 301 obtains an output resolution that is set according tothe output mode obtained in step S1902.

Next, in step S1903, the CPU 301 makes a determination as to whether theoutput resolution obtained in step S1902 is 200 dpi. In a case where theoutput resolution is set to 200 dpi, the CPU 301 sets the number ofobtaining low resolution frame image data to four frames (step S1904).

Next, in step S1905, a document reading is performed with the scanner,so that four frames of low resolution frame image data are obtained.

Next, in step S1906, the image converter 317 corrects inclination of thelow resolution frame image data obtained in step S1905.

At this moment, the inclination angle θ of the obtained frame image datais a value that can be obtained when the area sensor 213 is mounted onthe reading unit 205 in an assembly step of the multifunction apparatusincluding this area sensor.

This inclination angle θ is held, as a value unique to the mountedapparatus, in a memory area in the multifunction apparatus.

An affine transformation is performed using the above-described angleinformation, so that the obtained, obliquely inclined frame image datais rotated. At this occasion, a rotation is made for an amount ofinclination with respect to the reference arrangement position of theframe image data. Thereby, the inclination of the frame image data iscorrected. Where a coordinate before the conversion is (x, y), acoordinate after the conversion is (X′, Y′), and a rotation angle (inthe present embodiment, the inclination angle of the area sensor) is θ,the frame image data whose inclination has been corrected can beobtained through the affine transformation processing as shown inFormula 1.

$\begin{matrix}{\left\lbrack {X^{\prime},Y^{\prime}} \right\rbrack = {\left\lbrack {X,Y} \right\rbrack\begin{bmatrix}{\cos\;\theta} & {\sin\;\theta} \\{{- \sin}\;\theta} & {\cos\;\theta}\end{bmatrix}}} & \left( {{Formula}\mspace{14mu} 1} \right)\end{matrix}$X′, Y′: Coordinate position after the conversionX, Y: Coordinate position before the conversion

The frame image data obtained through the above-described affinetransformation becomes the low resolution frame image data whoseinclination has been corrected.

It should be noted that the method for correcting the inclination is notlimited to the affine transformation. Any method is applicable as longas the method can correct the inclination of the frame image data.

It should be noted that this processing is necessary in a case where theframe image data without any inclination can be obtained with the sensorcapable of obtaining the frame image data whose shift of the positionread by the sensors adjacent to each other in the short-side directionis less than one pixel in the primary scan direction and/or thesecondary scan direction as shown in (b) of FIG. 14.

Then, in step S1913, the super-resolution processing is performed usingthe multiple frames of frame image data whose inclination has beencorrected, and the processings are terminated.

The super-resolution processing performed at this occasion will bedescribed in detail using FIGS. 20, 21.

FIG. 20 is a figure showing the low resolution image data used for thesuper-resolution processing and the image data after thesuper-resolution processing. FIG. 20 shows an original document and areference low resolution image data F0 and target low resolution imagedata F1 to F3 obtained by reading the original document with the areasensor. A dashed rectangle enclosing the document denotes an area fromwhich the area sensor reads the reference low resolution image data F0.A solid rectangle denotes an area from which the area sensor reads eachof the target low resolution image data F1 to F3.

In the present embodiment, a shift amount in the primary scan directionis denoted as “um”, and a shift amount in the secondary scan directionis denoted as “vm”. These shift amounts regarding the target lowresolution image data Fn (n=1 to 3) are denoted as “umn”, “vmn”. Forexample, as shown in FIG. 20, the target low resolution image data F1 isshifted in the secondary scan direction with respect to the target lowresolution image data F0, and the shift amounts thereof are denoted asum1, vm1.

Similarly, the shift amounts of the target low resolution image data F2,F3 are denoted as um2, vm2 and um3, vm3.

The shift amounts umn, vmn regarding each of the target low resolutionimage data Fn (n=1 to 3) are calculated based on the image data of thereference low resolution image data F0 and the image data of the targetlow resolution image data F1 to F3. The calculation employs apredetermined calculation method based on the inclination information ofthe area sensor previously memorized in the ROM 303.

In FIG. 20, the shift of each target low resolution image data isassumed to be in unit of one pixel and is shown accordingly in schematicform.

However, a phase shift of less than one pixel in the primary scandirection and the secondary scan direction occurs during the readingperformed by the area sensor according to the present embodiment. Theuse of this minute shift enables the image to be made into highresolution.

Thus, among each pixel constituting a super-resolution processing imageto be generated (hereinafter referred to as “a generated pixel”), thereexist pixels that exist in neither of the reference low resolution imagedata nor the target low resolution image data.

Such pixels are made into high resolution while performing a combiningby performing a predetermined interpolation processing using pixel datarepresenting pixel values of pixels existing around the generated pixel.The usable interpolation processing includes a bi-linear method, abi-cubic method, a nearest neighbor method.

For example, the interpolation processing through the bi-linear methodwill be described using FIG. 21. First, a nearest pixel 1802 located atthe nearest distance from the position of a generated pixel 1801 isextracted from the reference low resolution image data and the targetlow resolution image data. Then, in the target low resolution image datain FIG. 21, four pixels surrounding the generated pixel position aredetermined as surrounding pixels 1802 to 1805, and data values of thesurrounding pixels added with predetermined weights are averaged toobtain data value of the generated pixel from the following formula.f(x,y)=[|x1−x|{|y1−y|f(x0,y0)+|y−y0|f(x0,y1)}+|x−x0|{|y1−y|f(x,y0)+|y−y0|f(x1,y1)}]/|x1−x0∥y1−y0|

For example, a super-resolution image of twice as high resolution can beobtained as shown in FIG. 20 by repeating the above-described processingfor each generated position. It should be noted that the resolution isnot limited to twice, but can be various magnification ratios. Inaddition, it should be noted that the more data values of the multiplelow resolution image data are used for the interpolation processing, themore precise super-resolution image can be obtained.

On the other hand, in a case where it is determined in step S1903 thatthe output resolution obtained in step S1902 is not 200 dpi, the CPU 301makes a determination as to whether the output resolution is 300 dpi(step S1907). In a case where the output resolution is set to 300 dpi instep S1907, the CPU 301 sets the number of obtaining frames of lowresolution data to ten frames (step S1908) and proceeds to S1905.

On the other hand, in a case where it is determined in step S1907 thatthe output resolution obtained in step S1902 is not 300 dpi, the CPU 301makes a determination as to whether the output resolution is 600 dpi(step S1909). In a case where the output resolution is set to 600 dpi instep S1909, the CPU 301 sets the number of obtaining frames of the lowresolution image data to forty frames (step S1910) and proceeds toS1905.

On the other hand, in a case where it is determined in step S1909 thatthe output resolution obtained in step S1902 is not 600 dpi, the CPU 301makes a determination as to whether the output resolution is 1200 dpi(step S1911). In a case where the output resolution is set to 1200 dpiin step S1911, the CPU 301 sets the number of the obtaining frames ofthe low resolution image data to 100 frames (step S1912) and proceeds toS1905.

On the other hand, in a case where it is determined in step S1911 thatthe output resolution obtained in step S1902 is not 1200 dpi, the CPU301 determines that the output mode is invalid and proceeds to stepS1901.

<Output Mode Acquisition Processing>

The output mode acquisition processing described in step S1901 in FIG.19 will be described in detail. In the present embodiment, the outputresolution is determined by obtaining a scan resolution.

FIG. 22 is a figure for describing an outline of operation performingthe output mode acquisition processing. The control program realizingthe processings as shown in FIG. 22 is stored in the ROM 303 asdescribed above, and is executed by the CPU 301.

First, in step S2201, the user designates the scan resolution via theoperation section 13 serving as the UI, and the operation section 13having received this designation sends this designation to the CPU 301via the operation section I/F.

Then, in step S2202, the CPU 301 designated with this scan resolutionsets this scan resolution as the output resolution and terminates theprocessing.

FIG. 23 is a schematic diagram showing an example of the operationsection for setting the scan resolution, and the user can make settingswith a scan resolution setting menu 2301 using this.

In the present embodiment, the scan resolution is designated, but theoutput resolution may also be determined by previously setting themaximum resolution of the apparatus.

<Document Reading Processing>

The document reading processing described in step S1905 in FIG. 19 willbe described in detail. In the present embodiment, the number of framesof the obtaining frame image data is determined by controlling aneffective area of the area sensor.

FIG. 24 is a figure showing the area sensor in the scanner 11 used inthe present embodiment.

The area sensor shown as numeral 2401 is divided in the short-sidedirection into 100 pieces, and one divided piece is made one band and iscontrolled to perform reading as one line sensor.

In unit of this band, as many as 100 frames of 100 dpi frame image datacan be obtained.

Thus, in a case of step S1904 in FIG. 19, as shown by numeral 2402, theframe image data is incorporated using bands 0 to 3 as an actual inputimage data reading area. Band 4 and subsequent bands are masked.Similarly, in a case of step S1908 in FIG. 19, as shown by numeral 2403,bands 0 to 9 are made the actual input image data reading area. Bands 10and subsequent bands are masked. In a case of step S1910 in FIG. 19, asshown by numeral 2404, bands 0 to 39 are made the actual input imagedata reading area. Band 40 and subsequent bands are masked. In a case ofstep S1912 in FIG. 19, the frame image data is incorporated using allthe bands (2405) as the actual input image data reading area. As aresult of operation as described above, the number of frames ofobtaining frame image data can be determined.

In this way, the number of frames of the obtaining frame image data canbe controlled according to the output resolution, and the anglecorrection of the inclined image data can be performed using as manyimage data as the number of obtained frames (S1906).

Then, using those frame image data, the super-resolution conversion isperformed (S1913).

It should be noted that in a case where non-inclined frame image datacan be read as described above in S1905, it is not necessary to performthe angle correction processing in S1906.

As a result of the processings as described above, the number of framesof low resolution frame image data needed for the super-resolutionprocessing can be set according to the set scan resolution, and highresolution image data can be obtained by performing the above-describedsuper-resolution processing using the obtained frame image data.

In addition, the number of frames of necessary low resolution frameimage data is controlled based on the resolution needed during theoutput. Thus, data amounts can be reduced during reading.

In addition, the number of frames of low resolution frame imagesobtained during reading is previously limited. Accordingly, acalculation amount of the super-resolution processing can also bereduced.

Second Embodiment

In the first embodiment, the output resolution is determined byobtaining the scan resolution. The present embodiment describes a casewhere the output resolution is determined by designating the copy mode.

The same processings as the first embodiment will be illustrated withthe same reference numerals, and the outline description thereabout willbe omitted.

The present embodiment describes four modes as copy modes, namely, aphotographic paper photograph mode, a character mode, acharacter/photograph/map mode, and a printed photograph mode.

In the photographic paper photograph mode, halftone performance isregarded as important, and a setting is made to output at the resolutionof 300 dpi.

Similarly, in the character mode, the resolution is regarded asimportant, and a setting is made to output at the resolution of 1200dpi.

In the other modes, both of the halftone performance and the resolutionare mutually satisfied, and a setting is made to output at theresolution of 600 dpi.

The relationship between the copy mode and the output resolution can bearbitrarily set.

FIG. 25 is a figure for describing an outline of operation performingthe output mode acquisition described in step S1901 in FIG. 19. Thecontrol program realizing the processing shown in FIG. 25 is containedin the ROM 303 as described above, and is executed by the CPU 301.

First, in step S2501, the copy mode designated with the operationsection 13 is obtained. Next, in step S2502, a determination is made asto whether the obtained copy mode is the photographic paper photographmode. In a case where it is determined that the obtained copy mode isthe photographic paper photograph mode, the output resolution is set to300 dpi (step S2503), and the processings are terminated.

On the other hand, in a case where it is determined in step S2502 thatthe obtained copy mode is not the photographic paper photograph mode, adetermination is made as to whether the copy mode obtained in step S2502is the character mode (step S2504). In a case where it is determined instep S2504 that the obtained copy mode is the character mode, the outputresolution is set to 1200 dpi (step S2505), and the processings areterminated.

On the other hand, in a case where it is determined in step S2504 thatthe obtained copy mode is not the character mode, a determination ismade as to whether the copy mode obtained in step S2502 is thecharacter/photograph/map mode or the printed photograph mode (stepS2506). In a case where it is determined in step S2506 that the obtainedcopy mode is the character/photograph/map mode or the printed photographmode, the output resolution is set to 600 dpi (step S2507), and theprocessings are terminated.

On the other hand, in a case where it is determined in step S2506 thatthe obtained copy mode is not the character/photograph/map mode or theprinted photograph mode, the processing returns back to step S2501.

FIG. 26 is a schematic diagram showing an example of the operationsection for setting the copy mode, and the settings can be made with acopy mode setting menu 2601.

As a result of the processings as described above, the number of framesof low resolution frame image data needed for the super-resolutionprocessing is set according to the set scan resolution, and theabove-described super-resolution processing is performed using theobtained frame image data. Thus, high resolution image data can beobtained.

In addition, the number of frames of necessary low resolution frameimage data is controlled based on the resolution needed during theoutput. Thus, data amounts can be reduced during reading.

In addition, the number of frames of low resolution frame image dataobtained during reading is previously limited. Accordingly, acalculation amount of the super-resolution processing can also bereduced.

As a result of the processings as described above, the number of framesof low resolution frame image needed for the super-resolution processingis set according to the set copy mode, and the above-describedsuper-resolution processing is performed using the obtained frame imagedata. Thus, high resolution image data can be obtained.

In addition, the number of necessary low resolution image data iscontrolled based on the resolution needed during the output. Thus, dataamounts handled during reading can be reduced.

In addition, the number of frames of low resolution frame image dataobtained during reading is previously limited. Accordingly, acalculation amount of the super-resolution processing can also bereduced.

Third Embodiment

In the present embodiment, a case will be described where the outputresolution is determined according to an enlargement ratio and areduction ratio of the document.

It should be noted that the same processings as the first embodimentwill be illustrated with the same reference numerals, and the outlinedescription thereabout will be omitted.

In the present embodiment, an enlargement ratio of 141% for performingan enlargement printing upon enlarging A4 document into A3 document isused as a reference, and a jaggy in an enlargement processing is reducedby setting the output resolution in such occasion to twice as much asthe scan resolution.

On the other hand, a reduction ratio of 70% for performing a reductionprinting upon reducing A3 document into A4 document is used as areference, and a calculation amount in a reduction processing is reducedby setting the output resolution in such occasion to half of the scanresolution.

In the present embodiment, each output resolution is fixedly set. Butneedless to say, each output resolution may be arbitrarily set accordingto each enlargement and reduction ratio.

FIG. 27 is a figure for describing an outline of operation performingthe output mode acquisition described in step S1901 in FIG. 19. Thecontrol program realizing the processing as shown in FIG. 27 iscontained in the ROM 303 as described above, and is executed by the CPU301.

First, in step S2701, an enlargement reduction ratio X designated by theoperation section 13 is obtained. Next, the scan resolution is obtainedin step S2702. Next, a determination is made as to whether theenlargement reduction ratio X obtained in step S2701 is between 100% and141% (step S2703). In a case where it is determined in step S2703 thatthe enlargement reduction ratio X is between 100% and 141%, the outputresolution is set to twice as much as the scan resolution.

For example, in a case where the scan resolution is 300 dpi, the outputresolution becomes 600 dpi (step S2704).

When the output resolution becomes known as described above, processingsafter this are performed according to the flow shown in FIG. 19. Thus,in a case where the output resolution is set to 600 dpi, the processingproceeds from S1909 to S1910, and the number of frames of necessaryframe image data becomes 40 frames.

Thereafter, the processing proceeds to S1905, and the original documentimage is read. In S1906, 1913, the super-resolution processing, namely,image correction and high resolution processing, is performed.

On the other hand, in a case where it is determined in step S2703 thatthe enlargement reduction ratio X is not between 100% and 141%, adetermination is subsequently made as to whether the enlargementreduction ratio X is between 70% and 100% (step S2705). In a case whereit is determined in step S2705 that the enlargement reduction ratio X isbetween 70% and 100%, the output resolution is set to one-half of thescan resolution. For example, if the scan resolution is 600 dpi, theoutput resolution becomes 300 dpi (step S2706).

In this way, when the output resolution becomes known, the processingssubsequent to this are performed according to the flow shown in FIG. 19.Thus, in a case where the output resolution is set to 300 dpi, theprocessing proceeds from S1907 to S1908, and the number of frames ofnecessary frame image data becomes 10 frames.

Thereafter, the processing proceeds to S1905, and the original documentimage is read. In S1906, 1913, the super-resolution processing, namely,image correction and high resolution processing, is performed.

FIG. 28 is a schematic diagram showing an example of the operationsection for setting the copy mode, and the user can make settings with amagnification ratio setting menu 2801.

As a result of the processings as described above, the number of framesof low resolution frame image data needed for the super-resolutionprocessing is set according to the set enlargement reduction ratio,i.e., a scaling ratio during the output, and the above-describedsuper-resolution processing is performed using the obtained frame imagedata. Thus, high resolution image data can be obtained.

In addition, the number of frames of necessary low resolution frameimage data is controlled based on the resolution needed during theoutput. Thus, data amounts handled during reading can be reduced.

In addition, the number of frames of low resolution frame image dataobtained during reading is previously limited. Accordingly, acalculation amount of the super-resolution processing can also bereduced.

Fourth Embodiment

In the present embodiment, a case will be described where the outputresolution is determined according to an output layout of the originaldocument.

The same processings as the first embodiment will be illustrated withthe same reference numerals, and the outline description thereabout willbe omitted.

In the present embodiment, a reduction layout is set to 300 dpi in acase of 2 in 1 (two pages of the document is realized in one page), andthe reduction layout is set to 200 dpi in a case of 4 in 1 (four pagesof the document is realized in one page). Thus, the smaller an area ofone page becomes, the lower the output resolution is set to.

FIG. 29 is a figure for describing an outline of operation performingthe output mode acquisition described in step S1901 in FIG. 19. Thecontrol program realizing the processing shown in FIG. 29 is containedin the ROM 303 as described above, and is executed by the CPU 301.

First, in step S2901, layout information representing a layout settingvalue designated by the operation section 13 is obtained. Next, adetermination is made as to whether the layout setting value obtained instep S2901 is 2 in 1 (step S2902). In a case where it is determined instep S2902 that the layout setting value is 2 in 1, the outputresolution is set to 300 dpi (step S2903), and the processings areterminated.

On the other hand, in a case where it is determined in step S2902 thatthe layout setting value is not 2 in 1, a determination is subsequentlymade as to whether the layout setting value is 4 in 1 (step S2904). In acase where it is determined in step S2904 that the layout setting valueis 4 in 1, the output resolution is set to 200 dpi (step S2905), and theprocessings are terminated.

FIG. 30 is a schematic diagram showing an example of the operationsection for setting the reduction layout, and the user can make settingswith a reduction layout setting menu 3001.

As a result of the processings as described above, the number of framesof low resolution frame image data needed for the super-resolutionprocessing is set according to the set output layout, and theabove-described super-resolution processing is performed using theobtained frame image data. Thus, data amounts can be reduced duringreading capable of obtaining high resolution image data.

In addition, the number of necessary low resolution frame image data iscontrolled based on the resolution needed during the output. Thus, dataamounts handled during reading can be reduced.

In addition, the number of frames of low resolution frame image dataobtained during reading is previously limited. Accordingly, acalculation amount of the super-resolution processing can also bereduced.

Fifth Embodiment

In the first to fourth embodiments, during the document readingprocessing, the area sensor is divided at the constant band width, and alimitation is applied in units of bands, so that the effective area iscontrolled. In the present embodiment, the output resolution controlsthe band width of the area sensor, so that the number of obtained framesof low resolution frame image data is determined.

FIG. 31 is a figure showing the area sensor in the scanner 11 used inthe present embodiment.

The area sensor as shown by numeral 3101 is arbitrarily divided in theshort-side direction, and multiple line sensors are bundled to betreated as one line sensor and controlled to perform reading.

Thus, in a case of step S1904 in FIG. 19, as illustrated by numeral3102, the area sensor is divided into four bands, and the image data areincorporated as bands 0 to 3. Similarly, in a case of step S1908 in FIG.19, the frame image data is incorporated as bands 0 to 9 as shown bynumeral 3103.

In a case of step S1910 in FIG. 19, the frame image data is incorporatedas bands 0 to 39 as shown by numeral 3104.

In a case of step S1912 in FIG. 19, the image data is incorporated asall bands (3105). As a result of the operations as described above, thenumber of frames of obtaining frame image data can be determined.

In this way, the area of the area sensor is divided into the number offrames of obtaining frame image data.

As a result of the processings as described above, the number of framesof low resolution frame image data needed for the super-resolutionprocessing is set according to the set scan resolution or the set outputresolution, and the entire area sensor can be effectively utilized. Inaddition, the number of frames of low resolution frame image dataobtained during reading is previously limited. Accordingly, acalculation amount of the super-resolution processing can also bereduced.

In addition, as described above, the number of times of reading in thesecondary scan direction is increased during the original document imagereading, and the number of times of sampling per unit time is increased.Thus, the number of obtained frame image data obtained in the short-sidedirection of the sensor can be increased.

Thus, by controlling the timing of sampling, the number of obtainedframe image data obtained in the short-side direction of the sensor canbe controlled.

Other Embodiments

Furthermore, the present embodiment can be applied to a system includingmultiple appliances (for example, computer, interface appliance, reader,printer, and the like) and can be applied to an apparatus including asingle appliance (image processing apparatus, printer, facsimileapparatus, and the like).

The object of the present invention may also be accomplished by causinga computer (or a CPU and an MPU) in a system or an apparatus to read outand execute a program code in a memory medium memorized with the programcode realizing a procedure of the flowcharts described in theabove-described embodiments. In this case, the program code itself readfrom the memory medium realizes the functions of the above-describedembodiments. Hence, the program code and the memory medium in which theprogram code is memorized constitute the present invention.

Examples of the memory medium for supplying the program code include afloppy (registered trademark) disk, a hard disk, an optical disk, amagnetic-optical disk, a CD-ROM, a CD-R, a magnetic tape, a nonvolatilememory card, a ROM, and the like.

Further, the program code read out by the computer can also be executed.The present invention may also include a case where the functions of theabove described embodiments are accomplished by causing an OS (operatingsystem) or the like running on the computer to perform a part or all ofthe actual processings based on instructions of the program code.

Further, a program code read out from the memory medium may be writtento a memory provided on an expansion board inserted into a computer orin an expansion unit connected to the computer. In such occasion, a CPUor the like provided in the expansion board or the expansion unitperforms a part or all of the actual processings based on instructionsof the program code, and these processings realize the functions of theabove-described embodiments.

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all such modifications and equivalent structures andfunctions.

This application claims the benefit of Japanese Patent Applications No.2007-330976, filed Dec. 21, 2007, and No. 2008-317281, filed Dec. 12,2008, which are hereby incorporated by reference herein in theirentirety.

The invention claimed is:
 1. An image processing apparatus, comprising:an area sensor unit configured to read from an original document image aplurality of frames of image data having a shift of less than one pixel;an output resolution acquisition unit configured to obtain an outputimage resolution at which the original document image read by the areasensor unit is output; an acquisition frame number control unitconfigured to control a number of frames read by the area sensor unitaccording to a result of the output resolution acquisition unit; acorrection unit configured to correct an inclination of the frames ofimage data controlled by the acquisition frame number control unit; anda high resolution conversion unit configured to perform interpolationprocessing using the plurality of frames of image data whose inclinationis corrected by the correction unit to obtain image data in a resolutionhigher than a resolution during reading, wherein the output resolutionacquisition unit determines the output image resolution according to atype of the original document image.
 2. An image processing methodcarried out in an image processing apparatus having an area sensor unitthat reads from an original document image a plurality of frames ofimage data having a shift of less than one pixel, the image processingmethod comprising: obtaining an output resolution at which the originaldocument image read by the area sensor unit is output; controlling anumber of frames read by the area sensor unit according to a result ofthe obtaining step; correcting an inclination of the frames of imagedata controlled in the controlling step; and performing interpolationprocessing using the plurality of frames of image data whose inclinationis corrected in the correcting step to obtain image data in a resolutionhigher than a resolution during reading, wherein, in the obtaining step,the output image resolution is determined according to a type of theoriginal document image.
 3. An image processing apparatus, comprising:an area sensor unit arranged with a first sensor and a second sensoradjacent to the first sensor, the first sensor and the second sensorbeing arranged in such a manner that a reading position of an originaldocument image is shifted by less than one pixel between the firstsensor and the second sensor among a plurality of sensors; an outputresolution acquisition unit configured to obtain an output imageresolution at which the original document image read by the area sensorunit is output; an acquisition frame number control unit configured tocontrol a number of frames of image data read by the area sensor unitaccording to a result of the output resolution acquisition unit; and ahigh resolution conversion unit configured to perform interpolationprocessing using the frames of image data controlled by the acquisitionframe number control unit to obtain image data in a resolution higherthan a resolution during reading, wherein the output resolutionacquisition unit determines the output image resolution according to atype of the original document image.
 4. An image processing methodcarried out in an image processing apparatus having an area sensor unitarranged with a first sensor and a second sensor adjacent to the firstsensor, the first sensor and the second sensor being arranged in such amanner that a reading position of an original document image is shiftedby less than one pixel between the first sensor and the second sensoramong a plurality of sensors, the image processing method comprising:obtaining an output image resolution at which the original documentimage read by the area sensor unit is output; controlling a number offrames of image data read by the area sensor unit according to a resultof the obtaining step; and performing interpolation processing using theframes of image data controlled in the controlling step to obtain imagedata in a resolution higher than a resolution during reading, wherein,in the obtaining step, the output image resolution is determinedaccording to a type of the original document image.
 5. A non-transitorycomputer-readable memory medium for storing a program that, whenexecuted by a processor, causes an image processing apparatus having anarea sensor unit that reads from an original document image a pluralityof frames of image data having a shift of less than one pixel to executean image processing method comprising: obtaining an output imageresolution at which the original document image read by the area sensorunit is output; controlling a number of frames read by the area sensorunit according to a result of the obtaining step; correcting aninclination of the frames of image data controlled in the controllingstep; and performing interpolation processing using the plurality offrames image data whose inclination is corrected in the correcting stepto obtain image data in a resolution higher than a resolution duringreading, wherein, in the obtaining step, the output image resolution isdetermined according to a type of the original document image.